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(54) Apparatus and method for modifying a compressed video signal 



(67) A compressed video stream (10} mat exhibtis 
{©m^rar redundancy {e.g., an MPSG-compiiant video 
stream) is daeorftptessed into an uncompressed do- 
maid representation {1 4). and the uncompressed dt^ 
m>.t ^ , w , M b, iL 

cnangns spa-as rtsJationsops in tne representation 
{ 13}. Sue ; - .tiding ^-atei- 

mafte lathe arscompressod domain representation, re- 
moving watermarks, from she uncompressed domain 



representation, altering watermarks in the .uncom- 
pressed domain representation, and adjusting hue and 
t- - unc xasss epresentation. 
modified repre on jresse 
tag the same motion vecto $ th< » - used to produce 
the original video stream {20}. Thus, the uncompressed 
domain tepresentatton is recompressed without recom- 
puting new motion vectors, thereby avoiding computa- 
it-iiy inter ? pre ss } 
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Description 

OToiessicn ac- 
prt scneme 
that expioits temporal redundancy The invention also 5 
; elates to copy prstsstkir; -of a compressed data stream 
COOOaj High capacity disc ptayers such as DVD play- 
ers can provide access to wide screen movies having 
-4 a-<" 1 1 " c S M£! v 

i > • f - „ >fu>>e v t ?c 

entertainment to the home. 

|Cf003] A fyjHsngth movie may be stored on a single 
DVD. This is made posst'bte because ;he DVD has a 
high storage capacity. A single DVD can presently store 
severai gigabytes o? data. Tit is is atso made possible ?s 
r ^sed according 

i > in MPS g >cneme. Modern corrs- 

jfession sche C ; a" wo ,oe csata i-'cf 

age requirements fcy factors o; -5 to 80 without a con- 
siderable iocs in th» qeaiity of :ho r;sconsiructed video * 
sequences, 

f0004| However, the advent of read'write DVO drives 
tor ocmp.it-.-:; nas nase ,< eocene - copy the DVD re- 
tcacss onto bianfe OVOs, thereby providing the oppor- 
tunity to inake ana nthodsed copses of the DVD releases, ss 
Sr. the absence of a copy protection schema, qualify of 
these unauthorised cosies is comparable to the quality 
of the DVO rateasassoicSfd stores. Therefore, unauthor* 
5£ 3P >anc n of OVO releases pose a 

ehafienee to artists, producers and copyrights owners 30 

■' ^ - -■ :ac ,.: v ser«sand infe- 

lacteal property rights. 

:[000S| Wmm copy protection schemes are availa- 
"d die r ,ha ior cf 
ft; a DVD releases. For example, rasiSent watemarte 3$ 
can fee 'imprinted* in a compressed video sequence, A 
: - : - 1 - " ■ ^ • < sr< to determine 

wt cit ;r; disc C ) ill 
played. The watermark might indicate whether the disc 
s oo npai c - s th 2 j«c sequence stored thereon, <e 
if too watermark indicates thai She video sequent 
- x i DVD player de- 

tects an 'incompatible "written* disc and shots down. 
|00rj6j However, modttytng the vioeo sequence By 



h as MPS 3 is tie- 



been compressed wo an M * ... 

JOOOSJ The present invention seeks to provide g 

M~~3 - 
mm According to an aspect c,< the cfdsant tm t 
there is pi u , o asm 

[0010} in the press ec sf.Dodsm.-o < 
stream that was compressed according to a me 
compensated prediction scher. 
1 compressed into an aneompre; 

lion Thetrod- s - as sthen 

modified without subsi n 

ships in the representation. The modified representation 
is then recompressed into a second data stream. The 
motion vectors in the first data stream are eseti tn the 
second data stream. 

[0011 J An embodiment o; the pi ass: s tnv or tion s tie- 
scribed, by way of example only, with reference to the 
accompanying drawings, in which: 

Figure 1a ts an itiustsa > > 
an MPEG tv' , t c t ■ 

shown in a display order: 

Figure lb is an iBustratton of sxsmpt&y frames of 
an MPEG-ccmpiiaet video stream, the frames being 
shown in a coding order 

Figure 2 is a block diagram o! a preferred embodi- 
ment of system; 

Figure 3 is a s v - - a ro ' <. 
^ (modified a pixei stoma ep sent 

J t, 4 - K' x. 

computer for modifying art tvSPEG compiian! video 
stream: and 

Figore 5 ts a flowchar! cf an alternative method of 
modifying a video stream. 

[0012} As Shown in the drawin gs tor purposes of lus- 
tration. She preferred embodiment is oesenbad in con- 
nection with a system for modifying a video stream such 
as an MP EG -compliant video stream. The system ah 
lows the video stream to be modified in a manner that 
is fess computationally intensive than simply decom- 
pressing the MPEG 3e i- ncompressed 
tion, modifying tne video in the unccm- 



a"y (( « pressed representation, and 



the vi ieo s« - s <iy aeen compressed into 

an MPEG-co-m , sam Decompressing the 
MPEG-cDrnpfsam video rtsrsam into spatiai-pixe! domain 
representations, moditying the pixel domain represen- 
tations and then re-co nj i th ecSfied px&t do- 
main represen j jr<S,1PEv3<orrp!antvtd 
eo stream is es :p ta i i \ intensive 
[OOOTj There might aiso bo a need to alter an existing 
watermark after the watermark has bsen added to the 
1 :o i to 'smove 

tne oxistirio watermark from the video sequence. Hera 
too, modifying ;he Video sequence isnoroputaticnafly in- 
tensive, especially if the video sequence has already 



than generating a eorn- 
pieteiy new MPEG-cor . r , am Lncfeo ,ng 

new motion vec s \ the tysten ailows 

waterrrvarks to be added ta, or removed from, a video 
stream in a lees computationally intensive manner 
so [0013] in the pa graph* ttt foitow de-scrip or of 
She preferred embodiment oS system will be provided 
The system wiil be described mainly in connection with 
a spatial pixel domain • < prions ots 

video sequence in the spaliat Pomam and an MPEG- 

limit' j 
[0( 14} i s , 

may be described as a hierarchy o! units A video se- 
quence is made up of a group o! pictures ("SOPs"). 
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Each GO- is made up to a numbs - tfigt. is pic 
turns, and each pictcm is made up of a numb-scof slices. 
Esc h slice indue oeks, and each 

reacioijtocii netodesabfe Ea h^PFGmac- 

sobtock typicafiy contains one or more 8x8 oiocks of pix- 
els. 

[Q01S] MPEG is a block motion-compensated predic- 
tion schema thai reduces both temporai and spatial re- 
tfcndaocres snhen - ue cs MPEG cod 

ing is genaraff? psffortned as idiom. The pictures o? a 
video sequence are oodso as intraframas (*i tames' 1 }, 

orward predicts r tea" i i o, -a motion 

to-prediction - "}. £a i GOP typlcgiiy 

efresbes the 5 , igins rs ar 

" » T t i si < , . < t 

GOP sscarms ghi, i jcttom fasbion. 

Every block (i.e., 3x8 biock) s codad. No prediction is 
used in the j frame. 



^ * 1 chniques ^ the video str eam 10 

A simpfsfiad description is as f oScws. The Stocks in each 
of She remaining pictures of the SOP are scanned in a 
SefMo-righi, top-io-bc-aom fashion. A btock is compared 
to similar stead sections of adjacent pictures, ft a good 
match for the biock is found, only the difference between 
the biccK an on and the vector point- 

ing » the matching section -I.e., the motion vector) are 
processed. The Terence' batween the block and the 
matching section is commoniy referred to as a "differ- 
ence bloc^ > or analysis is compu- 
tahonatiy intensive. 

[0017] Typicaiiy. one motion vector is specified for 
aach forward-predicted « backward-predicted bfock, 
and two motion vectors are specified for each bidirec- 
tional predicted black. Thus, aach P frame may include 
motion vectors that are related to a previous frame, 
white each S frame may include forward and backward 
motion vectors thai are refeied to adjacent frames. 
[001 8] so P frames, each biock may ba coded using 
forward proc c \ schniques InB 

frames, each c j forward predic 

iive coding techniques, backward predictive coding 
toon r t x'c coding techniques, 

or marrams coding techniques. 
lOOIB] Tht f < m a GOP to contain a 

fisxlcfe number o? pictures, dot typical GOP sizes tangs 
between fifteen and thirty five frames. The I, P and B 
frames for each OOP are arranged in fiaxlble orders. A 
ypica PBpstte 1 1n a display Ofde is si >wr n F gu 
la. and a typical IPS pattern in a coding order is shown 
srt Figure 1 b. Data ;s placed in a video siream according 
to the coding order. 

[0020] The original intratrame bioeks and difference 
v < e p idt =i s i transformed 

iron he spat ! t quer^y domain MPEG 

uses Discrete Cosine Transform {"OCT") coding to 
•i -f<?inieb!ccks 

into. 8xa biocks of DCT coefficients. 



[0031} The resuiii 19 DC! coes'f stents are quantized. 
Quantization usually results in a sparse representation 
of the data, i.e., usually most of the amplitudes of toe 
; in' red OCT tors areeq 
5 1 v. I fficie r 

varrabie-fengtii ens coding 
motion vectors am also compressed tiSiOg vananis- 
length coding techniques 

[0022] An MPEG -compliant video stream includes 
»e encoded DCT ccrto ts on vectors 

The Dol c ,*\ . .. moto so; < grwpec 
in GbPs. The video stream also includes header Infor- 
mation for each of the SOPs. 
[0023} Figure 2 shows a system -to iif 
« .MPEG-compliahtviOsosfreartto'C Fhe M 

ant video stream 10 is supplied to an MPEG decoder 
14, which completely dscomprasses hs video sf team 
10 into a pixel domain representation of the MPEG-en - 
coded video seq decon t ss 



stream 1 C is >. ^ 
motion vectors, the quantized coefficients are recon- 
stfticted, and the matron veotors and reconstructed DCT 
Cpeffict&nte are used to reconstroct the pixel domain 
as representation of the via os - - p< - 

fosming the com en one* 1 - - c o it . . iooc- stream 
10, the decoder 14 stores the motion vectors ^ & buffer 

(0024} Aconventionsl video editor 18 can then modify 
3e tha pixel domain representation of the video sequence. 
However, the modification should bs made without sub- 
stantially apijc do 
- Ttay nciude. 

withoat limttafion, the fottowing: adjusting the intensity 
& andhueiflfheptxeSdxi. 

art ifacl to She pixel domain representation, removing an 
existing artifact from the pixel domain representofion, 
reducing noise, and altering an existing artifact In the 
pixel dornt eprasi 
*> J0025} For example, an existing watermark could ce 
removed from a video seq^c to, -c- 
be added to disc urage ut pv *g of the vid- 

eo sequence. A watermark could be added to the pixel 
domain repress it 1 ■ v ne The 

« watermark could bed st toto - - t > nom 
noise that is invisible to the viewat * ei ; • .• 3ac st 
quence is being played back. However, ihs waterms-k 
can be detected by a copy ptcKeciswi-enabted sysiern. 
The watetmark > * ,< ^ 

so a disc is compatible with the video sequences stored 
theraor if a die :< <' a copy of the 

video sequence is ins - ic on r - 

abled player, and the v ^ , <*■ - n ,jci > 

seqaencs should > iotsc. thsp 

mil m pfa> s - jbtshuicf 

fOOSS] A re-encoder 20 recompfasses the rriodifis^o 
pixel domain rnoiocentatlon nack Into an MPEG -com- 
pliant video s! re, - ssn st pen orm 
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/ectors s - be 
ors the s 
compliant video strea 
sentation (tie 'modsfi 



modii sx doi «i epf se 
s-enccdar 20 re -uses She motion 
buffer 16. Using the buffered rno- 
r 20 generates an MPEGV 
e modified f epf€ 
id' vsdso stream 22). Thus, rec- 
ompression of the modified :•»<«• domem representation 
is Seas computationally intensive then the compression 
that was used to produce the angina; video stream 10. 
Adti i ona i j js ^ompi/-* 

tionsity intensive as the decompression performed by- 
She decoder 1 4. The modified video stream 22 may then 
be used to create a master DVD, or the modified video 
stream 22 may be stored an a disc 
[0027] Reference is now made to Figure 3, which 
shows how a P or S trams is coded by the re-encoder 
SO. "ma re-encoder 2D scans blocks in the picture (biocK 
-0 and acces s - n t -notion vectors 

mm in She buffer 18 (ssock 502}. The re-encoder uses 
the motion vectors -n find the matching sections in ad- 
jacent frames and to recompute .the dfiierence blocks 
(block 104). In this manner, the metier, vectors are re- 
- • • -.as are recomputed with- 

out having to recompute the motion vectors. 
[0028] New OCT coefficients are recomputed for the 
mac-obbeks {block 108). The new OCT coefficients are 
quantized {block 108), and the quantized coefficients 
are run length encoded and varfebie iengtn encoded 
(biock no) Hi i d>a variable 

length eocodao {biock 112). Header information is then 
< Jded bices 114 if educe he modified vk str< > 
22. 

£0028}' f? the same sequence of frames is kept, the 
headers and certain l, P and B frames may also be buff- 
ered in the buffer iS and rs-ussti oaring re^ompres- - 
sion. Since the re-encoder 20 does not re-compute the 
motion voc o - iptiant stream 

in a less com? ut -ve manner. 

I 00 ^] v mbina- 
4 ^ of h i i » ent such as a « 

software-based video editor and a hardware-based en- 
coder and re*encodai he sftemattVe, the sy stem may 
; 

|0O3"S] A soft hown in Figure 

4. Acompy ; . nrocesscr 202 and mem- < 

J £04 a oded a < cutaWe 

instructions Whs- executed the instructions instruct 
she piocesso- \" c , . , 

froms penpheraf device. & oarcs drive, ins intarnet). and 
then perfct - 1 dee edrt.ng and re- $ 

bove "he memo y 204 may 
also provide the buffering. The modified video stream 
22 may then b« c ti jvice 206 fe.g 

a hard drive or DVD pfeysr) or an output card 208 {e.g. 
* - <■ em or net wo K ca d* foi transmission to another s 

[0032] "h. iso" v > , .- - ; \ t Decompresses 
"ev jmin - 'is ;onta ion a 



mentation. Ree 



5 compression that 



lows the spatial domain representation to be modified. 

and then m-..si 

modified spas® don 

sion of the modified pixel domain 
5 oompyfatfesnajjy intensive then f 

proauo -s r c 

put nail - i pressicn that 

produces the pure! do i Thus e 

sysfemailowsa video stream tobe modified in a manner 
w that (s more com - simpiy d 

compressing a video stream, making modifications in 

the spatial pixel domain, and men generating a com- 

pfeteiy new > id 

Consequently, watermarks can be added loan MPEG* 
m compiiant video stream, removed from the video 
stream, or altered ;n a tt=ss ' - , - e-sr-e 

manner. Similariy. the hue and intensity of the video 
stream can be adj^'eo 
5ive manner. 

so [0033] Whenffof . > , ivrti?^ : 
huaorinttoo-. i stlai change, th< 

buffered motion vectors mgnl no longer ne ideal. Ccn- 
sequentiy, the difference blocks mighf not code quite as 
weit. However, too r , . t o j , 

« stight loss in guaiity. 

J0034} The system could be eppfisd to varices MPEG 
coding standards such as MPEG-1 and MPES-2, as 
weti as otner coding etiindards mm roiy on motton-com- 
pensatton (e g. motion JFT. ^ - / - - V . - 

» the method is not limited fo each cod ngsia 

exampte ot the MPEG-coT,;:.i, : - „ jiraam to was 
provided merety to faciistafe an understanding of the de- 
scribed embodiment. 

{0035] The method could be used m conjunction with 
5 any compression schema that uses moteo vectors or 
equtvaiants thereof. 

[0036] The method is not limited to OV0 players. The 
example of the DVD player wee also provided to facili- 
tate an understanding o? the preferred embodiment. 

5 [0037] Moreover, the invention s not flmited to the 
specific embodiments oq-: ribeda i : jstrated above. 
For exampte, the re-encooa: coaid generate the motion 
vectors independentf> h a re-encod- 

ar would parse the video sires toregr stht 

5 oa\ ' v.sw 3fe the buff* 

an oft-she-sheif decoder to be used. 
[00383 The method is not iimfted to a spatial pixel do- 
main representation. For msiance. the video sequence 
may De nsodified a fregur * it g 

' Figure 5, the video stream may be decompressed into 
a f reque ncy domain r ep - ^ - n t ye ng me vat 
iabie length encoding, the run length encoding, etc 
(block 500} T, i0 f roquo- ~ s tor t entafion ma> 
then fas modified by directiy modifying the reconstructed 

;T coefficients 1 e modified 

s esentatksn , *- < " i < 

tfiad rtraam by q en's (hloch 

;, a ength < abie length en- 
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coding in® quantized coefficients pock 936). The mo- domain repmsensasra and wherein the re-encoder 

lion vectors m m\ mpd'f?tsd; they are simply variable operable to compress a modified .'fwqywcy. do- 

lengir coded (bforis SDS). Thus, she motion vectors in main reptesentatsm 

She ofiginai video mmsm am used in the modified 

stream. 

[0039] Thus, she steps ;:- ; voived *n performing the de- 
compression and recompression may or may not in- 
clude fa* ig OCT and irtvs 2 DC nsforms instead, 
the dec " , r - f wit depend 

•spent type - s edited 

[0040} The oisciosaros in Un ited States patent appli- 
cation no. 09/1 §8,052, from which this application 
daims -priority, and in ths abstract accompanying this 
application arc incorporated herein: by reference. " 



C tains 

t. Apparatus (12) to? stowing a firs: motion vector- 
omp s s ' 1 -cm odified i « >c so 
par&lus comprising: 

a decoder tot) responsive 10 the first stream, 
for providing a?; unccmprssscd domain repre- 
^ ■. t ' r- , , fi - : , , , f .?£ 
inc. oscornpo! as ;on c? the Brst data stream; and 
are-encoder (2 .. tho motion vec- 

tors and a mods* j.nccr resseddoma 
resemato, - 

tor-compliant oats stream (22). ths re-enccsder *> 
using pturaiity of motion vectors in the sec- 
ond stream, 

2. The apparatus of claim 1 , comprising a video editor 
(205 k- ".compressed represertta- 3$ 
tion, the vtdao editor being operabie to provide »he 
modified representation !o the re -encoder. 

3. The separated? eiatm 2; wherein ths decoder, vid- 
eo editor and re-anccderare realised by a computer 
{200} including s processor (202) and memory 
(204), ;he memory being encoded with a plurality of 
instructions thai., wnen executed, instruct the proc- 
essor So decompress ins stream into ths uncom- 

" essed jo 5 r, c< -! ^ n moon- « 
pressed domain seprot-entatrcr: to be mo&ftea. and 
re-compress the roodsfiad representation into the 
modified data stream. 

4. Ths <jpp 3 - -qq^ so 
operable to yes the pisrslify of motion vectors to 
provide an uncompressed apaPa; domain repressn- 
ition, and v ncoder is operable to 

i ai? " v -e p a vectors to compress a 

modified spa „ - cental on. ss 

5. The apparatus of ciaim p. wherein the decoder is 
operable to provide an uncompressed frequency 



The apparat - a buffs? (IS) 

for bulterini erwto * 
decomprer s - i ths ro -en- 

coder being operabfa to use the battered motion 
vectors for the genetsiior} of the second stream 

The apparatus oi claim 1, wherein the decoder is 
operable *c par o m cseccd i d "sg tc - 
MP£G standard, whsre-n the re-encoder is opera- 
ble to perform encoding according to an MPEG 
standard, and wherein the first and second data 
streams are M v - ~v - 1 x - o - -raans 
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FIGURE 3 
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FIGURE 5 
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{54} Apparatus and method for modifying a compressed video ssgna? 



(57) A compressed vkfao stream ( 1 0) that exhibits 
temporal redundancy (e.g., an MPSG-conpSant video 
stream) is decompressed into art uncompressed do- 
main representation (14), and the uncompressed do- 
main representation is modified without substantially 
ehsnging spatial relationships in We representation 
08}. Such r.'x , r ^ snciude adding water- 
marks to the uo rsj in representation , re- 
n-ovlr-.g watermarks tram the uncompressed domain 



epresentation t j ate 
pressed domain representation, and adjusting hue and 
intensity ir> the uncompressed domain representation. 
The modified representation Is then reeompressed us- 
ing me same motion vectors that were used to produce 
the original video stream {20). Thus, the uncompressed 
domain representation is reeompressed without reeom- 
pufing new motion vectors, thereby avoiding computa- 
tionally intensive processing. 
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